This invention relates to a system and methods for setup and measuring the position and orientation (pose) of transponders. More specifically, for training the user to manipulate the pose of the transponders through a movement trajectory, while guided by interactive and sensory feedback means, for the purposes of functional movement assessment for exercise, and physical medicine and rehabilitation.
Known are commercial tracking and display systems that employ either singularly, or a hybrid fusion thereof, mechanical, inertial, acoustical or electromagnetic radiation sensors to determine a mobile object's position and orientation, referred to collectively as pose.
The various commercial tracking systems are broadly classified by their relative or absolute position tracking capability, in which system the pose of a mobile object is measured relative to a fixed coordinate system associated with either combination of receiver(s) or passive or active transmitter(s) housing mounted on the user. The tracking system's components may be tethered with obvious inherent movement restrictions, or use wireless communication means to remotely transmit and process the information and allow for greater mobility and range of movement.
Typically these tracking systems are utilized for biomechanics and gait analysis, motion capture, or performance animation and require the sensors to be precisely mounted on the joints. Various means of presenting the tracking information in a visual display are employed, such as Heads-Up Display (HUD), that provide occluded or see-through visibility of the physical world, or Fixed-Surface Display (FSD), such as computer desktop monitors, depending upon the simulation and immersive quality required for the application. The application may require various degrees of aural, visual, and tactile simulation fidelity and construct direct or composite camera views of the augmented or three dimensional (3D) virtual reality environment to elicit interactive user locomotion and/or object manipulation to enhance the user's performance and perception therein. The tracked object may be represented in the virtual environment in various forms, i.e., as a fully articulated anthropoid or depicted as a less complex graphical primitive. The rendering strategy employed depends upon the degree of photo realism required with consideration to its computational cost and the application's proprioception requirements.
Tracking technologies possess certain inherent strengths and limitations dependent upon technology, human factors, and environment that need consideration when discussing their performance metrics. Regardless of differentiating resolution and accuracy performance benchmarks, many implementations suffer from varying degrees of static and dynamic errors, including spatial distortion, jitter, stability, latency, or overshoot from prediction algorithms. Some human factors include perceptual stability and task performance transparency, which are more subjective in nature. And environmental issues such as line-of-sight, sensor attachment, range, and multiple-object recognition, need to be considered when selecting the optimal technology for the most robust application development. Irrespective of the intrinsic strengths and weaknesses of the tracking technology employed, ultimately the user's satisfaction with the system's utilization and efficacy, including the production of reliable, easily understood, measurable outcomes, will dictate the overall success of the device.
This invention's system and methods facilitates biomechanical tracking and analysis of functional movement. In the preferred embodiment, this invention is low cost, robust, easy to deploy, noninvasive, unobtrusive, and conveys intuitive and succinct information to the user to execute movement properly and provides performance indicators of said movement for feedback purposes. One feature of the present invention provides for an interactive tracking system because the sensor functionality, or referred to herein as active transponders or transponders, is integrated with local user input control, and real-time sensory interfaces on the same device. The transponder is a wireless communication and monitoring device that receives a specific signal and automatically responds with a specific reply. In one embodiment, the invention provides functional movement assessment based upon the relative measures of limb pose with respect to two positions defined by the transponders. The transponders can operate independently or work in unison to process and share computational tasks and information between the local databases. This decentralized, distributed processing scheme allows the configuration and coordination of the training session, and processing and analysis of the measurements to occur without requiring expensive auxiliary computer and display systems to manage the same, and without relying on costly software development of complex synthetic environments for visualization purposes. Also, the user can manage the applications and performance databases off-line on a remote computer system with Internet connectivity to customize and configure the system parameters in advance of their session.
The present invention is designed to provide such system and methods for high-fidelity tracking or registration of the poses of active transponders and engage the user to purposely manipulate the transponders' pose along a prescribed or choreographed movement trajectory in order to train and assess functional movement capability. In the preferred embodiment, the system is comprised of two subsystems: (1) a subsystem comprised of one or more active transponders, which, in its most sophisticated implementation, responds to periodic requests from another component of the system to radiate or transmit a signal for purposes of absolute position tracking; processes an embedded inertial sensor for relative orientation tracking and absolute tracking refinement; and provides an essentially real-time aural, visual, and tactile sensory interfaces to the user, and (2) a subsystem comprised of a centralized position processor system or unit and receiver constellation unit, collectively referred to as the processor unit, which is essentially a signal processor that synchronizes the transponders' periodicity of radiating signal and other operational states; collectively receives and processes the radiated signal; iteratively calculates the transponders instantaneous pose and convolution, thereof; and continually exchanges this information, and its' analysis thereof, with the transponders and/or auxiliary host computer system in essentially real-time via a combined wireless and tethered communication means. This real-time bi-directional exchange of information allows for proper transponder identification, coordination, and the accurate measurement of pose, thereof, and timely actuation of the sensory interfaces for optimal user regulated closed-loop control.
The transponder is broadly classified by its level of hardware and software configuration that define its scope of intelligence, sensory support, and configuration. The degree of intelligence is determined by its capability to locally access, process, and modify the database. Further, either transponder classification can be sub-classified by its manipulative requirements. In one embodiment, where multiple transponders are used, a principle transponder is consciously and deliberately moved along the reference movement trajectory, while a subordinate transponder serves as an anchor or secondary reference point elsewhere on the locomotion system whose kinematics are not necessarily controlled by the user's volition.
An interactive transponder, preferably, has significant intelligence; supports relative and absolute tracking capabilities; provides complete sensory stimuli support; provides for functional enhancement through attachment of modular, extension pieces; and provides a user display and input system to control the training session. In the preferred embodiment, the interactive transponder is primarily held in the hand to facilitate more complex user input and greater sensory intimacy. Conversely, in another embodiment, the fixed transponder has limited intelligence; supports only the absolute pose tracking capability; provides no sensory stimuli support; and is usually mounted to a fixed site on the limb or trunk.
A combination of transponder deployment strategies may be required depending on the training session's objectives, such as two interactive transponders grasped by each hand; or alternatively, an interactive transponder, and a fixed transponder attached to the limb or trunk; or lastly, two fixed transponders attached to the limb(s) and/or trunk.
In one embodiment, this invention proposes to elicit movement strategies based on the deployment of at least two transponders that define the endpoints of a movement vector whose relative translation and rotation is measured and evaluated for the assessment of functional movement capability, including but not limited to, limb range of motion and its control thereof, limb strength conditioning, and overall proprioception and hand-eye coordination skills, and overall body movement. This registration system measures a single movement vector whose endpoints are comprised of an anchor point, i.e. one that is located in a less dynamic frame of reference, e.g., such as the trunk or abdomen, and another more distal location fixed on or held by a limb or extremity, e.g., the hand, arm, or leg. As this movement vector is translated and rotated through space by the act of the user modifying the pose of the principle transponder in concert with the reference movement trajectory, the vector's length will expand and contract relative to the proximity of principle transponder with respect to the subordinate transponder. The vector's length conveys unique and explicit information regarding the user's movement efficiency and biomechanical leverage. For example, by attaching a fixed subordinate transponder at the hips and a fixed principle transponder on the upper arm, the biomechanics of the act of lifting a box or similar object can be elegantly qualified. If the user assumes a poor lifting technique, i.e. legs locked with the trunk severely flexed with head down and the arms stretched out beyond the basis of support, the vector's length would consistently be measured longer than compared to a good lifting technique, i.e., legs bent at knees with the back straight, head gaze up, and arms close to body. Also, the measurement(s) of higher-order derivatives derived from numerical mathematical processes of a reference point described by the vector would provide additional indication of movement control or smoothness. In summary, one embodiment of the present invention is comprised of:
a means to create a single movement vector whose endpoints are defined by the locations of at least two transponders, wherein, the expansion and contraction of the vector's length is calculated, analyzed, and reported in essentially real-time;
a means to create a single movement vector whose endpoints are defined by the locations of two transponders, wherein, a representative point along the vector length is referenced and its higher-order derivatives are computed by mathematical numerical processes, wherein the result is calculated, analyzed, and reported in essentially real-time; and,
a means to correlate said vector's length and at least one other measure consisting of a higher-order derivative, to the reference movement trajectory, wherein the result is calculated, analyzed, and reported in essentially real-time.
A registration system for practical functional movement applications should clearly convey information to the user regarding his movement quality while he performs the task, without compromising or distracting from said execution by unnecessary head movements or change in eye gaze and normal focus. Poor visualization strategies that distract the user are ineffectual for promoting heads-up, immersive interaction, and the alphanumerical information it imparts often can not be consciously processed fast enough to elicit corrective action. This system provides for both a local, standalone sensory interface as a primary feedback aid, or alternatively, an interface to a remote fixed-surface display for greater visualization and simulation capabilities. The visual stimulus could be modulated to warn of range violations, or provide signals for purposes of movement cadence and directional cueing. A principle interactive transponder is typically hand-held, which is naturally in close proximity to the user's aural and visual sensory field during most upper extremity movements, or, conversely, the visual stimulus may be viewed through a mirrored or reflective means if not in optimal line-of-sight. A remote fixed-surface display might augment the immersive quality of the user's experience by providing control of a view camera of a simulated computer environment, and display of the transponders and/or interactive objects' static or dynamic poses within the computer display's skewed through-the-window perspective projection.
In summary, one embodiment of the present invention is comprised of:
a means for modulating an embedded luminescent display organized and oriented into a directional-aiding pattern, by varying its degree of intensity and color, or other physical characteristics, to provide a visual display stimulus. This sensory interface is excited at a rate, repetition, or pattern proportional to the pose error of the transponders' movement trajectory compared to the reference movement trajectory;
a means to view said visual display stimulus with the aid of a mirror(s) or other reflective means;
a means for the real-time projection of sound or speech commands through an audio device to provide warning, alarm, instructional, and motivational aid, and/or additional cueing upon encroachment of static and dynamic limit/boundary conditions defined by the reference movement trajectory;
a means for real-time tactile feedback including, but not limited to, modulation of the rotational properties of a vibrator motor proportional to the pose error of the transponders' movement vector compared to the reference movement trajectory;
a means for combining the excitation of said stimuli proportional to the pose error of the transponders' movement vector compared to the reference movement trajectory; and,
a means to coordinate the real-time, periodic parametric update and modulation of the stimuli imparted by the sensory interfaces within the transponders from a processing unit by means of a wireless communication link.
This invention addresses the need for an intuitive, interactive method to instruct, create, and deliver a movement trajectory command without necessarily relying on pre-programmed, regimented movement trajectories. The registration system can be configured via remote setup at the principle transponder to pre-record and choreograph a free-form movement trajectory of the principle transponder with the intent of the user mimicking the same said path. This impromptu learning modality can expedite the session down time between different users and movement scenarios, and accommodate users' high anthropometric variability in range of movement. In summary, one embodiment of the present invention is comprised of:
a means is to provide a movement trajectory learning modality that allows the user to calibrate and create the desired endpoints, midpoints, and/or total reference movement trajectory through user programmer entry of an input device resident on the transponder;
a means to process and save a movement trajectory using a computationally efficient Catmull-Rom spline algorithm or other similar path optimizing algorithms to create control points along key points of the movement trajectory that define the optimally smoothest path intersecting the control points;
a means to provide database management by a processing unit via a wireless communication link or, alternatively, through user data entry of an input device resident on the interactive transponder; and,
a means to access, edit, and store the program and/or databases to nonvolatile memory operably coupled to the principle transponders for the purpose of automating the creation, delivery, storage, and processing of movement trajectories. Customized user programs and databases would be downloaded from a central repository or relevant website in advance of the training session to the transponder from the user's home location via the Internet or other convenient locales having networked Internet access, and transported to the systems remote physical location, and uploaded into the system's memory, and executed as the application program.
This a priori process of remote selection, download, and transfer of programmatic content and database would minimize the user's decision making and input during product utilization by offering only relevant and customized programming material of their choosing targeted for their specific exercise, fitness, or rehabilitation goals. Performance data could be saved indefinitely in the database's nonvolatile memory, until an upload process was performed through the said network so the database could be transferred to another location for purposes of, but not limited to, registration, processing, archival, and normative performance evaluation, etc.
An exemplary list of specific data structures contributing to or affecting the means for automating the creation, delivery, storage, and processing of movement trajectories described below may be stored within the non-volatile memory of the transponder or position processor which may use high-density serial FLASH, although other types of memory may be used such as SmartMedia, Compact Flash, etc. Additionally, the memory device interface should not be limited to internal, but may include external media devices, such as USB FLASH Key or other portable media means, that may have inter-operability with other computerized devices. The data structures may include:
Modulation & Feedback Thresholds/Triggers Properties—the aural, visual, tactile interfaces require threshold settings which determine their excitation or stimulation characteristics. These settings can be derived from previous performance data or defaults determined from normative data, or modified in real-time, by algorithmic methods including moving averages, standard deviations, interpolation based upon goal-oriented objectives, etc.
Normative Performance—performance data collected over a large population of users through controlled studies, that is distilled down into specific user categories based upon certain demographics that the user may compare and rank his/her results. This data may be initially embedded within the transponders or position processor non-volatile memory and may be augmented or modified automatically or by user volition when connected to the Internet.
Competitive Ranking—applications which have a predominate point goal-oriented purpose would allow access to a global ranking file archive accessed through the Internet or automatically via updated executive files. This ranking file would be created through an analysis of user participation and publishing of his/her results through Internet Web-based services.
Downloadable Executive Programs & Configurations—new software programs, including new features, enhancements, bug fixes, adjustments, etc., could be downloaded to the transponder through an Internet connection. Graphics images would be stored in compressed or uncompressed binary forms, i.e., bitmap, gif, jpeg, etc. These new programs could be transferred to any suitable computerized position processor unit located at a remote facility via the transponder's wireless link. Therefore, the user's transponder is the node that establishes the portable network, capabilities of the system, not necessarily the computerized position processor.
Custom Menu Interfaces—specialized activities may require more advanced (or simplified) interfaces dependent upon the users' cognitive abilities and interactive specificity. This menu may include interactive queries or solicit information regarding, the user's daily goals, subjective opinions or overall impression of the activity and ones performance which could be incorporated in the Motivation Index described below.
Report Generation Tools and Templates—XML, HTML or other authoring language used to create documents on the Web that would provide an interactive browser-based user interface to access additional performance data analysis and report generation tools and templates that may not be available or offered with the standard product.
Custom Performance Algorithms—certain application-specific performance analysis may require dynamically linked algorithms that process and calculate non-standard or specialized information, values, units, physical measurements, statistical results, predictive behaviors, filtering, numerical analysis including differentiation and integration, convolution and correlation, linear algebraic matrices operations to compute data pose and scaling transformation, and proprietary types. One example of a proprietary type is Motivation Index, a composite numerical value derived from a weighted average of statistical performance indicators and subjective user input including relative scoring improvements, conformity to ROM pattern, lengthy activity access duration, high access rate, relative skill level improvement, daily goal achievement, etc., that could represent the overall level of enthusiasm and satisfaction, the user has for a particular activity.
Range of Motion (ROM) Pattern Generator—the ROM pattern requires some key control points to be captured along the desired trajectory and stored in order that the algorithm can calculate an optimally smooth path, in real-time, during the comparative analysis phase.
ROM Pattern Capture & Replay—the ROM pattern can be saved to memory in real-time by discrete position samples versus time depending upon the resolution desired and memory limitations and later played back on the transponder or remote display for analysis.
Activity Specific Attributes—includes Reps/Sets, Duration, Pause, Heart Rate Limits, intra-activity delay, level, point scalars, energy expenditure, task-oriented triggers, etc., and other parametric data that controls intensity, execution rate and scoring criteria for the activity.
Instructional Information—textual, graphical, or animation-based instruction, advice, coaching, activity description, diagramed transponder deployment and intra-device connectivity, etc. that facilitates the intuitiveness, understanding, and usage of the system. The form of instruction may include music files saved in various formats, including Wave, MP3 or other current or future audio data compression formats, and video files saved in MPEG or other current or future video data compression formats.
Real-time Data Management—proprietary data management protocols that reside above the communication driver layer that manage the real-time, synchronous and asynchronous exchange of data between transponder(s) and position processor. This would provide an essential real-time sharing of activity data, analysis, and feedback stimulus thresholds, or coordination of multiple transponder configurations, or for a collaboration of same or different user requirements to complete a similar activity objective.
This invention addresses the need for adaptability of the registration system to different movement measurement scenarios. In one embodiment, it utilizes a versatile, modular configuration and mounting of the transponders onto the user. The efficient deployment of the transducers between different users' and from task to task requires a universal mounting scheme to provide consistent localization and pose of the transponders at the desired measurement sites on user's body. Also, to compensate for the receivers' finite tracking volume when stationary, the receiver constellation unit may be mechanically modified to optimize its tracking properties by conveniently repositioning it in closer proximity to the expected transponders movement trajectories and line-of-sight, thereof. In summary, one embodiment of the present invention is comprised of:
a means to quickly and efficiently alter the location of the transponders using a fastening system designed to quickly attach and dispose various forms of transponder assemblies;
a means to augment the physical properties, i.e., weight and length, of the principle transponder with adjunct electro-mechanical components that provide variations in biomechanical leverage for isotonic and isometric utilization; and,
a means to allow the user to manually alter the geometry and pose of the receiver constellation unit to facilitate an optimal tracking location based upon collectively maximizing the ultrasonic source's energy received at the transducer interface.
This invention addresses the practicality and robustness of the registration system when used in either indoor or outdoor environments, and especially when the tracking volume likely contains potentially occluding objects, i.e., uninvolved limbs or clothing, that become potential sources of competing, reflected paths. The preferred embodiment of the registration system utilizes the time of flight (TOF) measurement of ultrasonic acoustic waves due to its immunity from interference from the visible and near-visible electromagnetic spectrum and its superior ability to overcome most multi-path reflections problems by simple gated timing of the initial wave front. Upon command from the processor unit, the transponders produce a few cycles burst of ultrasonic energy and the transducers of the receiver constellation unit are stimulated and mechanically resonate accordingly, upon the wave front arrival. The processor unit's analog signal processing circuits transform the mechanical energy into electrical signals that resemble tapered sinusoidal waveforms, which another electronic circuit triggers upon using an adaptive threshold technique which, in turn, the processor unit detects and calculates TOF timestamps indicating the wave front arrival. In the preferred embodiment, the system overcomes the ultrasonic technology's intrinsic challenge of precisely triggering on the same waveform location and provides consistent unambiguous trigger detection by complementing the adaptive threshold technique with a software timestamp correction algorithm, which includes in part, a digital over-sampling and averaging timestamp algorithm, a relative timestamp correction scheme utilizing a predictive algorithm of higher-order Taylor series based derivatives, and an absolute timestamp correction scheme that minimizes the range error based upon discrete biasing of timestamps.
Further, in the preferred embodiment, the processor unit utilizes the absolute and relative trigger timestamps in a multi-modal trilateration algorithm for the measurement of three-dimensional (3D) translations and rotations of the transponders. The primary trilateration calculation is derived by an application of Pythagoream theorem involving a point position solution based-upon range measurements from at least three (3) points, versus the well-known triangulation method which uses bearing angles of two cameras of known pose. Additionally, the system's main accuracy limitation is mostly affected by the temperature variability of outdoor environments and its influence on the speed of sound in air value. This algorithm mitigates this problem by mathematically computing the speed of sound every analysis period provided at least five (5) receivers and a transponder synchronizing means are utilized. If the integrity of the synchronizing signal is temporarily compromised, the system automatically employs a variation of the trilateration algorithm that uses the last known speed of sound value.
In the preferred embodiment, the maximum update rate, and hence the major contributor to the latency of the position calculation, is determined by the typical acoustical reverberation, typically between 20 to 100 ms, encountered in an indoor environment. Since the transponders are held or fixed on the user's body and, therefore, are mobile, the TOF measurements will experience an additional latency effect. A Kalman filter is used as a prediction/estimation strategy to minimize and compensate for the latency effect. The prediction algorithm uses a higher-order Taylor series based derivatives and augmentative inertial sensor data. Its predictive refinement is dependent upon predefined models of expected movement conditions. Because functional movement is episodic, having periods of stillness interspersed with bursts of motion activity, a Multi-modal filtering strategy is preferably employed to handle the unpredictable jerkiness at the start of motion and relatively predictable, smooth motion afterwards. In summary, the preferred embodiment of the present invention is comprised of:
a means to detect the same carrier wave cycle of ultrasonic energy using a software correction algorithm requiring multiple, consecutive TOF acquisitions as input for the digital over-sampling and averaging algorithm, the calculation of a higher-order numerical differentiation of the past and current TOF information as input for the predictive algorithm of higher-order Taylor series based derivatives used for the relative TOF correction, and a measurement of the intra-pulse time intervals of consecutive TOF acquisitions as input for the absolute TOF correction scheme that minimizes the range error based upon selective biasing of the TOFs;
a means to utilize a dual matrix formulation of the trilateration algorithm, and a calculation strategy thereof, which decision is dependent upon the integrity of the system's communication link, synchronization condition, and the desired measurement accuracy; and,
a means to coordinate the information transfer between transponders and the processor unit so that their contribution to the resultant movement vector calculation can be measured without intra-signal interference.
These goals will be attained by such system and methods that are comprised of the user's interaction described by the following steps as set forth as the preferred embodiment:                a. Authenticate user access and open user session from a local or remote database;        b. Setup user training session, i.e., workload limitations, measurement criteria, and audio/visual/tactile stimuli;        c. Select training program and configure its options;        d. Deploy the transponders as instructed to predefined locations of users locomotion system to create at least one transponder movement vector;        e. Calibrate the transponder movement vector to establish its reference pose;        f. Create a movement trajectory using learn mode, if required;        g. Initiate the start of session;        h. Determine the instantaneous pose of transponder movement vector relative to its reference pose from a periodic temporal iteration of this step;        i. Perform qualitative and quantitative statistical analysis of accumulated measured poses of the transponder movement vector relative to the pattern of instantaneous poses defined by the reference movement trajectory;        j. Update the major transponders sensory interfaces to modulate said system parameters in a periodic temporal iteration of this step;        k. End the session once program objectives have been obtained;        l. Analyze the results by interacting with local and/or remote databases;        m. Provide numerical, graphical, and/or animated information indicating desired performance measurements.        